#90 #78
These issues mentioned above all stem from the same bug. Although some time has passed, I encountered the same problem and found an effective solution. I hope this helps the next person who comes across this.

The root cause is that the video scene you input has excessive changes, causing the optical flow algorithm to fail to find any matching keypoints between adjacent frames. Strangely, several other optical flow algorithms have already implemented effective handling for this scenario—specifically, assuming no pose change for frames where pose estimation fails. Only the default method applySparseOptFlow lacks this implementation. Therefore, the solution is straightforward: replace the following code:

matchedKeypoints, status, err = cv2.calcOpticalFlowPyrLK(self.prevFrame, frame, self.prevKeyPoints, None)

to:

try:
    matchedKeypoints, status, err = cv2.calcOpticalFlowPyrLK(self.prevFrame, frame, self.prevKeyPoints, None)
except:
    print('Warning: find transform failed. Set warp as identity')
    # Store to next iteration
    self.prevFrame = frame.copy()
    self.prevKeyPoints = copy.copy(keypoints)

    return H

This perfectly resolves the issue. It instructs the algorithm to use the predefined identity transformation H matrix for frames where matching fails due to significant mid-process changes. Simultaneously, it preserves this frame for continued matching downstream. This ensures the optical flow algorithm resumes normal operation immediately once scene changes subside, preventing persistent errors.

Ideally, you should now encounter only one or two printed warnings, with no further errors occurring.